Imaging device, imaging system, and imaging method

ABSTRACT

An imaging device includes: an image sensor that photoelectrically converts subject light to generate image signals; a storage portion that sets a second area within a first area so as to correspond to a specific area specified by a setting value for setting an extracting range of an image based on the image signals; a timing adjustment unit that adjusts the timing at which the image signals are read from the image sensor and written to the storage portion and the timing at which the image signals are read from the storage portion; an image conversion processor that performs predetermined processing on the image based on the image signals read from the image sensor; an output unit that converts processed image signals with continuous scanning timing into image signals of a predetermined format and outputs the image signals to a display unit.

FIELD

The present disclosure relates to an imaging device, an imaging system,and an imaging method which are ideally used in an in-vehicle camerathat captures and displays images of areas that are difficult for adriver, for example, of an automobile to see.

BACKGROUND

In the related art, techniques of displaying images captured by acompact camera mounted in an automobile vehicle, a railroad vehicle, orthe like on a display device installed, for example, near the driver'sseat are known (for example, see JP-A-2002-19556 and JP-A-2010-33108).By capturing the images of areas that are difficult for the driver tosee during driving using such a camera and displaying the capturedimages on a display device, such areas can be changed to areas that arevisible to the driver. For example, in the case of automobile vehicles,the rear or the like of a vehicle is often set as the areas that aredifficult for drivers to see taking a case in which the vehicle movesbackward to park into consideration. Moreover, at the intersections of anarrow road, it is difficult to recognize vehicles approaching from theleft and right sides.

Moreover, an imaging system which uses a monitoring device that capturesand displays images output by a plurality of cameras in order to displaythe surroundings of a vehicle is known. This imaging system enhances thevisibility of a plurality of images using a PinP (Picture in Picture)presentation function of displaying images captured in differentdirections on the same monitor screen. Another in-vehicle imaging systemis known which converts and combines parts of a whole image using an ECU(Electronic Control Unit) or the like for processing images output by acamera to thereby perform a PinP presentation using a function ofarranging separate screen areas within a screen. In this imaging system,after the images output by the camera are taken into an image memory ofthe ECU through wiring harness, the plurality of transmitted images arecombined together. Thus, the definition of images displayed on anin-vehicle monitor depends on the bandwidth of the transmission system.Therefore, the images are taken into the image memory of the ECU and theconversion images are combined together in a state where the resolutionof the transmitted images is decreased.

FIG. 18 shows an example of an image 100 of the surroundings of avehicle, captured with an in-vehicle camera of the related art.

In the related art, a camera system is known in which an in-vehiclecamera is disposed on the front of a vehicle to display the video on theleft and right sides of the vehicle to a display unit so that the usercan observe the areas on the left and right sides of the vehicle as wellas on the front side. Moreover, a camera system is known in which acamera uses an anamorphic lens or the like of which the optical axis ofa lens is designed to be asymmetrical so that the user can observe theareas on the left and right sides of a vehicle as well as on the frontside. In the former camera system, the images captured by three camerasdisposed on the front and the left and right sides of the vehicle aretaken, and the images are combined and displayed by an imagepresentation function used in a camera control ECU or a navigationsystem.

SUMMARY

However, when driving a vehicle across a road at a right angle orpulling the vehicle out from a parking garage, it may be difficult forthe driver to observe other vehicles or pedestrians approaching from theleft and right sides from the driver's seat. Since an in-vehicle cameramounted on a vehicle needs to capture a wide range of areas, a fish-eyelens, an anamorphic lens, and the like are used. Thus, if an imagecaptured by these lenses is displayed as it is, since objects located atthe periphery of the image are displayed in a small scale, the drivermay overlook dangerous objects or obstacles during driving.

Moreover, in the imaging system of the related art, it is necessary toattach a plurality of cameras to the left, right, front, and rear sidesof a vehicle. Furthermore, separate image combination processors areneeded to combine the images captured by the respective cameras. Thus,constructing an imaging system incurs a lot of effort and is expensive,and such a system is not widely used. In addition, when a camera isconfigured by combining a plurality of lenses, the lens system has acomplex configuration, and the camera becomes expensive and large. Thus,the mounting position of the camera is limited. Furthermore, the use ofa wide-angle camera of the related art to just display a wide-angleimage has a problem in that it is difficult for the driver to sense thedistance to an approaching vehicle due to a large distortion of thewide-angle camera. Therefore, it is very dangerous for the driver todrive the vehicle trusting the visually sensed distance since it is fardifferent from the actual distance.

Moreover, in the imaging system of the related art, since the imagedefinition is determined by the transmission capacity of the harness,the image definition will decrease when the image is enlarged by the ECUdisposed at the rear stage of the camera. Furthermore, since the ECUneeds to take all images transmitted from the camera into a memory once,it is necessary to mount components for image combination as well ascomponents for image storage on the imaging system. Therefore, thenumber of components needed for constructing an imaging systemincreases, making the imaging system complex and expensive.

It is therefore desirable to enable a driver (user) to easily see imagescaptured from the left and right directions in relation to a movingdirection together with an image of a local area that the driver focuseson with a particular attention on the same screen at the same time.

In an embodiment of the present disclosure, subject light isphotoelectrically converted to generate image signals.

A second area is set within a first area in a storage portion so as tocorrespond to a specific area specified by a setting value for settingan extracting range of an image based on the image signals.

The timing at which the image signals are read from the image sensor andthe image signals are written to the storage portion and the timing atwhich the image signals are read from the storage portion are adjusted.

Then, predetermined processing on the image based on the image signalsread from the image sensor is performed. The frame rate of writing theimage signals is changed in units of frames by the adjusted timing, anda one-directional image signal corresponding to an image obtained byimaging in one direction, among the image signals input from the imagesensor is written to the second area in a predetermined frame period. Onthe other hand, a left-directional image signal corresponding to animage obtained by imaging the left side of the one direction is writtento a first split area split from the first area in a frame perioddifferent from the frame period of the one-directional image signal. Aright-directional image signal corresponding to an image obtained byimaging the right side of the one direction is written to a second splitarea split from the first area in a frame period different from theframe period of writing the left-directional image signal to the firstsplit area.

The one-directional image signal, the left-directional image signal, andthe right-directional image signal read from the first and second areasin accordance with continuous scanning timing are converted into imagesignals of a predetermined format and outputting the image signals to adisplay unit.

With this configuration, by appropriately arranging and storing imagesignals in the storage portion, it is possible to store image signalsread from the image sensor in one storage portion and output theone-directional image signal, the left-directional image signal, and theright-directional image signal from the storage portion as they are insynchronization with the continuous scanning timing.

According to the embodiment of the present disclosure, the imagingdevice has a function of arranging a child screen within an image outputby the imaging device having a PinP function and output the child screento the display unit. In the child screen, an image based on theone-directional image signal is displayed. Moreover, in a screenexcluding the child screen, the left and right-side images based on theright-directional image signal and the left-directional image signal aredisplayed. In this way, when monitoring the surroundings using awide-angle image, the user can observe the areas in the horizontaldirection as well as the area in the one direction at the same time bylooking at the same screen. Moreover, by improving the visibility, thedriver can detect the situation of the surrounding environment of thedriver's vehicle at an early stage.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an exemplary internal configuration ofan imaging device and a display device according to an exemplaryembodiment of the present disclosure.

FIG. 2 is view illustrating examples of the timing at which an imagesensor of the related art reads out image signals and the timing atwhich an imaging device of the related art outputs image signals.

FIG. 3 is a view illustrating an example of images disposed in a firstarea and a second area.

FIGS. 4A to 4C are views showing examples of a process of writing aconversion image in first and second areas.

FIGS. 5A to 5D are views illustrating an example of an image conversionprocess by an image conversion processor according to an exemplaryembodiment of the present disclosure.

FIGS. 6A to 6E are views illustrating an example of a process by atiming adjustment unit according to an exemplary embodiment of thepresent disclosure.

FIGS. 7A to 7E are views showing an example of timing adjustment whendisplaying a partial image in an enlarged scale in a PinP mode in anexemplary embodiment of the present disclosure.

FIGS. 8A to 8D are views illustrating an example (first case) of timingadjustment when displaying a partial image in an enlarged scale in aPinP mode in an exemplary embodiment of the present disclosure.

FIGS. 9A to 9D are views illustrating an example (second case) of timingadjustment when displaying a partial image in an enlarged scale in aPinP mode in an exemplary embodiment of the present disclosure.

FIGS. 10A and 10B are views illustrating an example of enlarging theleft and right images in both vertical and horizontal directions in anexemplary embodiment of the present disclosure.

FIG. 11 is a view illustrating an example of an image conversion andenlarging process by an image conversion processor converting andenlarging the left and right small images having been subjected totiming adjustment according to an exemplary embodiment of the presentdisclosure.

FIGS. 12A and 12B are views illustrating an example of a PinP overlappresentation wherein a central image is displayed so as to overlap withan image enlarged in the horizontal direction in an exemplary embodimentof the present disclosure.

FIGS. 13A to 13D are views illustrating an example when a partial imageis displayed in an enlarged scale in a PinP mode in an exemplaryembodiment of the present disclosure.

FIG. 14 is an exemplary top view of an automobile in which an imagingdevice according to an exemplary embodiment of the present disclosure isinstalled.

FIGS. 15A to 15C are views illustrating an example of an image (thewhole image is disposed on the upper part) of the surroundings of avehicle, captured with an imaging device according to an exemplaryembodiment of the present disclosure.

FIG. 16 is a view illustrating an example of an image (the whole imageis disposed on the lower part) of the surroundings of a vehicle,captured with an imaging device according to an exemplary embodiment ofthe present disclosure.

FIG. 17 is a view illustrating an example of an image (the whole imageis disposed on the upper part and expanded in the horizontal direction)of the surroundings of a vehicle, captured with an imaging deviceaccording to an exemplary embodiment of the present disclosure.

FIG. 18 is a view illustrating an example of an image of thesurroundings of a vehicle, captured with an in-vehicle camera of therelated art.

DETAILED DESCRIPTION

Hereinafter, modes for carrying out the present disclosure (hereinafteralso referred to as a present embodiment) will be described. Thedescription will be given in the following order.

1. Exemplary Embodiment (PinP Presentation Control: Example ofControlling PinP Presentation using One Image Storage Portion); and

2. Modified Example

<1. Exemplary Embodiment> [Exemplary Overall Configuration of ImagingSystem]

FIG. 1 shows an exemplary configuration of an imaging device 1 and adisplay device 2 as an exemplary embodiment of an imaging system 5according to the present disclosure. In the present embodiment, a casein which the imaging device 1 is applied to an in-vehicle cameraattached to the rear of an automobile vehicle (not shown), and thedisplay device 2 is applied to a display device of a car navigationsystem or the like mounted in the vehicle will be described as anexample. The imaging device 1 and the display device 2 are connected bya cable or the like (not shown) whereby the imaging system 5 is formedin which image signals are input from the imaging device 1 to thedisplay device 2, and control signals are input from the display device2 to the imaging device 1.

In the present embodiment, although an example in which the imagingdevice 1 is mounted on an automobile is illustrated, the mountingdestination is not limited to this. That is, the imaging device 1according to the embodiment of the present disclosure may be mounted ona railroad vehicle and the body of a vehicle or a ship's hull as avehicular machine such as, heavy equipment (for example, a movable cranecar), an industrial machine, or a yacht.

First, the constituent elements of the display device 2 will bedescribed. The display device 2 includes a display unit 21 and anoperation input unit 22. The display unit 21 is configured, for example,by an LCD (Liquid Crystal Display) or the like and is configured todisplay image signals transmitted from the imaging device 1 as an image.The operation input unit 22 generates control signals in accordance withthe content of a button operation by a user pressing a Select/OK buttonB1 for selecting display screen settings and supplies a setting valuefor an image to be selected by the imaging device 1 to a control unit14.

The operation input unit 22 further includes a value input button B2independent from the Select/OK button B1. With the value input buttonB2, the images of “local areas that the user focuses on with particularattention” are selected and set in accordance with the content of abutton operation by the user, and a unique presentation mode is set inadvance and stored. By doing so, the unique presentation mode can beinvoked by a simple operation of pressing the Select/OK button B1 asnecessary. Hereinafter, this presentation mode set in advance by thisdisplay method will be referred to as a “personal view mode”.

In particular, when the user long-presses the Select/OK button B1 andthe value input button B2 at the same time, the operation input unit 22generates an interrupt request signal to the imaging device 1 so as totransit to a setting menu screen for “personal view mode”. On thissetting menu screen, the user presses the Select/OK button B1 and thevalue input button B2 continuously (this button operation is a firstcommand input) to thereby generate control signals, whereby anextracting position and control setting values necessary for partialenlargement are determined and stored in a setting value storage portion15. Moreover, the determined position and setting values are supplied tothe imaging device 1, and setting necessary for the unique presentationin “personal view mode” is performed.

Moreover, when the pressing of the Select/OK button B1 is detected, theimaging device 1 in which the “personal view mode” is set by thedescribed method performs the following processes. That is, this buttonoperation is regarded as a second command input. Moreover, a controlsignal to read the image of a area set as the “personal view mode” isgenerated.

Although the “personal view mode” is an example of a presentation modefor making blind areas unique to the user easier to see, areas which areregistered in advance when designing or manufacturing an imaging devicemay be set and displayed in a PinP format.

Although in the example shown in FIG. 1, the display device 2 includesthe display unit 21 and the operation input unit 22 as separate units, adisplay device that uses a touch panel or the like in which the displayunit 21 and the operation input unit 21 are integrated may be used.

Next, constituent elements of the imaging device 1 will be described.The imaging device 1 includes an image sensor 11, an image qualityadjustment unit 12, an image conversion unit 13, a control unit 14, asetting value storage portion 15, a timing adjustment unit 16, a displaycontrol unit 17, an analog encoder (output unit) 18, and an areaselection unit 19.

The image sensor 11 is configured by a solid-state imaging device. Asthe image sensor 11, a CCD (Charge Coupled Device) image sensor, a CMOS(Complementary Metal Oxide Semiconductor) image sensor, and the like canbe used, for example. The image sensor 11 generates image signals byphotoelectrically converting subject light condensed by a wide-anglelens (not shown) disposed near the body of a vehicle or a ship's hull onwhich the imaging device 1 is mounted. The image signals obtained by theimage sensor 11 are supplied to the image quality adjustment unit 12.The image quality adjustment unit 12 converts the supplied image signalsinto digital image signals and performs image quality adjustment. As theimage quality adjustment, AGC (Automatic Gain Control), noise reduction,image quality enhancement, and the like are performed, for example. Theimage quality adjustment unit 12 controls the driving of the imagesensor 11. Moreover, the image quality adjustment unit 12 adjusts theimage qualities of images made up of a one-directional image signal, aleft-directional image signal, and a right-directional image signalwhich are to be written in first and second areas 132α and 132βdescribed later in accordance with the timing at which the image signalsare taken from the image sensor 11 in units of frames. In this case, therespective directional images are controlled to have different imagequalities. The timing is determined based on a readout timing signalsupplied from the timing adjustment unit 16. The image signals of whichthe image qualities are adjusted by the image quality adjustment unit 12are supplied to the image conversion unit 13.

The image conversion unit 13 is configured to include an imageconversion processor 131 and an image storage portion 132 and isconfigured to convert the image format into a standard output format,for example, the NTSC format. The image storage portion 132 includes afirst area 132α for storing a first image signal corresponding to awhole image based on the image signals input from the image sensor 11and a second area 132β for storing a second image signal correspondingto a partial image which is a part of the whole image. Moreover, thefirst area 132α is split into two areas, which are a first split area132α1 for storing a left-directional image signal among the first imagesignal and a second split area 132α2 for storing a right-directionalimage signal.

In the present embodiment, a “one-directional image signal” obtained byimaging an area in one direction in which the lens of the imaging device1 faces is used as the second image signal. Moreover, a“left-directional image signal” obtained by imaging the left side of theone direction and a “right-directional image signal” obtained by imagingthe right side of the one direction are collectively used as the firstimage signal. The image storage portion 132 sets the second area 132βwithin the first area 132α so as to correspond to a specific areaspecified by a setting value generated by the control unit 14 that setsan extracting range of the image based on the image signals. Moreover,the first area 132α is set in a portion of the image storage portion 132excluding the second area 132β.

In the imaging device 1, the first and second areas 132α and 132β areprovided in the image storage portion 132, and the area selection unit19 is provided so as to select an area to which image signals are to bewritten. A conversion image to be displayed on the entire screen of thedisplay unit 21 is stored in the first area 132α, and a conversion imageto be displayed in a small PinP screen is stored in the second area132β. By continuously reading out images in the image storage portion132 in accordance with the frame output timing, the images of the PinPformat can be output in a standard output format.

Specifically, the image conversion processor 131 extracts an image of aspecific area within the image based on the image signals supplied fromthe image quality adjustment unit 12 based on the control of the controlunit 14. Moreover, the extracted image is enlarged to an image havingthe size corresponding to the full screen of the display unit 21 of thedisplay device 2. In this case, the image conversion processor 131performs predetermined processing on the image based on the imagesignals read from the image sensor 11. Moreover, the image conversionprocessor 131 writes image signals to the second area 132β whilechanging the frame rate of writing image signals in units of frames bythe adjusted timing. This adjustment is performed on the one-directionalimage signal corresponding to the image obtained by imaging an area inone direction among the image signals input from the image sensor.

Moreover, the image conversion processor 131 writes the left-directionalimage signal corresponding to the image obtained by imaging the leftside of the one direction in the first split area 132α1 split from thefirst area 132α in a frame period different from the frame period of theone-directional image signal. Furthermore, the image conversionprocessor 131 writes the right-directional image signal corresponding tothe image obtained by imaging the right side of the one direction in thesecond split area 132α2 split from the first area 132α in a frame perioddifferent from the frame period when images are written to the firstsplit area 132α1. Image signals different from the image signals writtento the first area are written in a subsequent frame period and insynchronization with time-sequential frame readout timing. The extractedand enlarged images are output to the image storage portion 132. Theimage storage portion 132 also serves as a frame memory for maintainingthe image signals supplied from the image conversion processor 131 for apredetermined period.

As described above, the imaging device 1 includes the image conversionunit 13 which has a function of generating and outputting conversionimages from a wide-angle image and which stores an image of a partialdesignated area to be embedded in a part of an image area necessary foroutputting the whole image. Since the image taken from the image sensor11 through the wide-angle lens (not shown) of the imaging device 1 has alot of distortion, the image conversion unit 13 processes the imagesignals to correct the distortion. In this case, the specific areas thatare desired to be displayed in an enlarged scale are enlarged andindividually inserted into a designated address area as a child screenof an output image and are maintained in the second area 132β used as abuffer memory. Moreover, the image is output together with an embeddedimage in the rear stage of the camera during frame update, whereby thePinP function is realized.

The control unit 14 is configured, for example, by an MPU(Micro-Processing Unit) or the like. The control unit 14 generates thesetting value for “personal view mode” defined by the “first commandinput” that specifies the range of “specific area” based on the size andposition of a frame in accordance with the content of the control signalinput from the operation input unit 22 and stores the setting value inthe setting value storage portion 15. That is, the setting value servesas a setting value that designating the extracting range of the imagebased on the image signal supplied from the image quality adjustmentunit 12. The setting value includes a zoom factor when zooming thespecific area and XY coordinates of the specific area. Moreover, thecontrol unit 14 includes a counter for counting the number of pressingsof the value input button B2 of the display device 2 and a timer formeasuring time.

Upon receiving the second command input from the operation input unit22, the control unit 14 reads the setting value from the setting valuestorage portion 15 and outputs the setting value to the image conversionunit 13. Moreover, the control unit 14 generates a control signal foradjusting the output timing of image signals from the image sensor 11and the output timing of image signals from the image storage portion132 and supplies the control signal to the timing adjustment unit 16.Furthermore, the control unit 14 generates a control signal foroutputting the image signals converted by the image conversion processor131 to the analog encoder 18 and supplies the control signal to theimage conversion processor 131.

The area selection unit 19 and the timing adjustment unit 16 adjust thetiming of reading image signals from the image sensor 11 and writing thesame in the image storage portion 132 and the timing of reading imagesignals from the image storage portion 132 based on the control signalsupplied from the control unit 14. The details of the processes of thetiming adjustment unit 16 will be described with reference to FIGS. 6Ato 6E.

The display control unit 17 reads images stored in the image storageportion 132 and outputs the same to the analog encoder 18 in accordancewith the timing adjusted by the timing adjustment unit 16 based on thecontrol signal supplied from the control unit 14. Moreover, the displaycontrol unit 17 generates a frame line or the like for setting thespecific area and outputs the same to the analog encoder 18. In thiscase, the display control unit 17 displays a frame of which the size andposition are varied in accordance with the operation input from the useron the display unit 21 for displaying images based on image signalsoutput by an output unit. Moreover, the display control unit 17 displaysan image extracted as the specific area in the child PinP screen.Furthermore, although an example in which the display control unit 17displays the image extracted as the specific area in the full PinPscreen and displays the whole image in the child PinP screen isillustrated, the output PinP screens may be set in the opposite manner.

The analog encoder 18 reads the one-directional image signal, theleft-directional image signal, and the right-directional image signalfrom the first area 132α (the first and second split areas 132α1 and132α2) and the second area 132β in accordance with continuous scanningtiming. Moreover, the analog encoder 18 converts and outputs the imagesignals (the one-directional image signal, right-directional imagesignal, and left-directional image signal) into image signals of apredetermined format in a state where the whole image and the partialimages are associated with each other. This format may be the NTSC(National Television System Committee) format, for example, and theanalog encoder 18 supplies the image signals to the display unit 21 ofthe display device 2. The timing adjustment unit 16 adjusts the imageoutput timing so that images can be output in synchronization with the VSync signal of the NTSC output timing regardless of whether images areextracted or not. This adjustment is performed in order to adjust thetiming of reading images from the image sensor 11 and outputtingconversion images by delaying the entire image frame.

FIG. 2 shows examples of the timing at which an image sensor of therelated art reads out image signals and the timing at which an imagingdevice of the related art outputs image signals.

In this example, a case in which the imaging device 1 of the related artdoes not perform conversion for correcting image distortion, the PinPpresentation mode, extracting and enlarging of partial images, and thelike will be described.

The image sensor reads one line of image signals in each horizontalperiod (1H). All horizontal lines in the imaging area of the imagesensor are read during one vertical period (1V). Images taken from theimage sensor 11 are output to the outside of the imaging device as theyare without being subjected to image size conversion or the like.

An image storage portion of the imaging device of the related art doesnot have the first and second areas 132α and 132β corresponding to theimage storage portion 132 of the present embodiment. Here, since theimage quality adjustment unit and the image conversion unit performspredetermined processing, respectively, the timing at which the imagingdevice 1 outputs image signals is generally delayed by the same amount.As a result, it is not necessary to perform timing adjustment for eachlocal image area.

FIG. 3 shows an example of images disposed in the first and second areas132α and 132β.

A conversion image to be displayed on the entire screen of the displayunit 21 is stored in the first area 132α, and a conversion image to bedisplayed in a small PinP screen is stored in the second area 132β. Inthe following description, the full screen of the display unit 21 willbe referred to as a “main screen” and an area in which a partial imageis displayed will be referred to as a “child screen”. The imageconversion unit 13 already knows the range of areas in the horizontaland vertical directions of the child screen.

Whenever one line of images processed by the image conversion processor131 are written to the image storage portion 132, the image conversionunit 13 calculates the starting and ending positions at which the childscreen is displayed and writes the image signal of the child screen tothe second area 132β in units of lines. Here, writing of image signalsto the first and second areas 132α and 132β is performed based on thevalue of a child screen determination counter (not shown) of the imageconversion unit 13. The child screen determination counter is a counterthat counts image signals written in the horizontal direction for eachpixel and counts image signals written in the vertical direction foreach line.

Since the writing is also performed in the vertical direction, the imagesignal of the child screen is written to the second area 132β if thevalue of the child screen determination counter is larger than the linecorresponding to the display starting position of the child screen.Moreover, the writing to the second area 132β ends if the value of thechild screen determination counter is larger than the display endingposition of the child screen. Here, the image signal written to thefirst area 132α is an image signal of the present frame, whereas theimage signal written to the second area 132β is an image signal of theprevious frame. A process of outputting image signals of the present andprevious frames by splitting one image storage portion 132 into two areawill be described with reference to FIGS. 4A to 4C and FIGS. 5A to 5D.

FIGS. 4A to 4C are views showing examples of a process of writing aconversion image in first and second areas 132α and 132β.

First, as shown in FIG. 4A, a conversion image S to be displayed in asmall PinP screen is written to the second area 132β. The first area132α is a mask area in which no image is written. Subsequently, as shownin FIG. 4B, a conversion image F to be displayed in the full screen iswritten to the first area 132α. In this case, the second area 132β is amask area in which no image is written.

Then, the area selection unit 19 causes the full screen conversion imageF and the small screen conversion image S which are written in this wayto be alternately output from the analog encoder 18 in units of frameswhile decreasing the frame rate by ½. That is, the respective conversionimages are updated every two frames. By performing such a process, asshown in FIG. 4C, the full screen conversion image F and the smallscreen conversion image S which are images obtained at different timesare combined and output.

With such a configuration, it is possible to display an image capturedin the normal mode on the full screen of the display unit 21, forexample, and display an image of a specific area Pa (see FIG. 6A) basedon the personal view mode in the small PinP screen. By outputting theimages of both areas in each output frame, the user can observe anenlarged image of the specific area Pa while looking at the normal imagecaptured by the imaging device 1. Thus, the user can easily understandthe situation of the surroundings of the vehicle. Moreover, according tothe configuration of the present embodiment, it is not necessary toprovide a plurality of image storage portions 132 so as to correspond tothe number of areas without using a device for generating and combiningPinP images at the outside of the imaging device 1 and additionalmemories necessary for this operation. Therefore, it is possible todecrease the number of components of a vehicle surrounding monitoringsystem configured by the imaging device 1 and the display unit 21 and todecrease the cost of the system.

[Exemplary Image Conversion Process by Image Conversion Processor 131]

FIGS. 5A to 5D show an example of the timing at which images areconverted.

FIG. 5A show an example of the readout timing at which images are readfrom the image sensor 11.

In this example, images are read from the image sensor 11 in units offrames.

FIG. 5B shows an example in which symbols are assigned to images readfor each frame.

FIG. 5C shows an example of conversion images input to the imageconversion unit 13. Since images are processed by the image qualityadjustment unit 12 which is disposed between the image sensor 11 and theimage conversion unit 13, the images are input to the image conversionprocessor 131 later than the timing when the images are read from theimage sensor 11.

FIG. 5D shows an example of the image readout timing.

The image conversion processor 131 increases the frame rate of the wholeimage or the partial image extracted from the input images in units offrames twice so that the difference in the timings at which the wholeimage and the partial image are written to the image storage portion 132is one frame. In the present embodiment, the whole image is illustratedas a conversion image α-n and the partial image is illustrated as aconversion image β-n (where n is a natural number).

The image conversion processor 131 writes the whole image to the firstarea 132α and the partial image to the second area 132β. The areaselection unit 19 reads the whole image and the partial image for twoframe periods. Moreover, the difference in the timings at which the areaselection unit 19 reads the whole image and the partial image from theimage storage portion 132 is one frame. As a result, two kinds of PinPimages of which the frame difference is 1 are displayed on the displayunit 21.

As described above, since the output of the respective conversion imagesis updated every two frames, images obtained at different times arecombined and output. In this way, the image sensor 11 can output imagesof every frame without decreasing the frame rate. The output frame rateis 30 fps (frame/second) in the NTSC format, which generates the timedifference but does not cause any problem in practical use. Moreover,since the imaging device 1 outputs images, it is possible to constructthe imaging system 5 so as to output images continuously insynchronization with a reference vertical synchronization timing signalsuch as the NTSC analog output.

The image conversion processor 131 may write the one-directional imagesignal to the first area 132α and write the left-directional imagesignal and the right-directional image signal to the second area 132β.In this case, the analog encoder 18 displays the left and right splitenlarged images in the child PinP screen within the screen displayed bythe display unit 21, and displays an image obtained by imaging an areain one direction on the main screen.

[Exemplary Timing Adjustment Process by Timing Adjustment Unit 16]

FIGS. 6A to 6E to FIGS. 9A to 9D show examples of a timing adjustmentprocess by the timing adjustment unit 16 when displaying a partial imagein an enlarged scale.

FIG. 6A shows the timing at which image signals are read from the imagesensor 11 using a whole image 6.

In FIG. 6A, the timing at which the first pixel (a pixel at the left-topcorner of the screen) among the respective pixels of one frame is readis denoted by Ti1, and the timing at which the last pixel (a pixel atthe right-bottom corner of the screen) is read is denoted by Tie. Afterthe first pixel is read at the timing Ti1, the horizontal scanningperiod 1H begins, and one line of pixels up to the right end of thescreen are sequentially read. After reading of the first line, pixels onthe second line to the last line are sequentially read, whereby allpixels of one frame are read. That is, in the reading of image signalsfrom the image sensor 11 shown in FIG. 6A, a period calculated by(timing Tie)−(timing Ti1) is an effective 1V period.

FIG. 6B shows an example of a partial image 7 in an area set as thespecific area Pa in the “personal view mode”.

In FIG. 6B, the specific area Pa is depicted by a broken line, thetiming at which the pixel at the left-top corner of the specific area Pais read is denoted by Tin, and the timing at which the pixel at theright-bottom corner is read is denoted by Tim. A period calculated by(timing Tim)−(timing Tin) changes with the size of the specific area Pa.In the present embodiment, a partial image 7 in the specific area Pa isextracted, and the extracted image is output to the display device 2during the effective 1V period.

FIG. 6C shows an example of the timing at which images are output to thedisplay device 2.

In FIG. 6C, the timing at which the first pixel of the partial image 7in the specific area Pa is output to the display device 2 is denoted byTo1, and the timing at which the last pixel is output to the displaydevice 2 is denoted by Toe.

FIG. 6D shows an example of the timing at which the image sensor 11outputs image signals.

In this example, the first horizontal period in the 1V period is denotedby “1H” , the second horizontal period by “2H”, and the x-th horizontalperiod by “xH”.

FIG. 6E shows an example of the timing at which the imaging device 1outputs image signals.

Due to the image quality processing and image extracting processing, thetiming at which the imaging device 1 outputs image signals is generallydelayed. However, since it is necessary to output the partial image overthe 1V period, the image is maintained in the image storage portion 132used as a buffer memory.

When outputting the partial image 7 in the specific area Pa shown inFIG. 6B over the effective 1V period expressed by (timing Toe)−(timingTo1), it is necessary to output the partial image 7 corresponding to thespecific area Pa in the effective 1V period with a delay. In addition,since a delay associated with the image quality adjustment processing bythe image quality adjustment unit 12 is also added, the timing at whichthe image signal is output from the imaging device 1 shown in FIG. 6D isdelayed by a predetermined period from the timing at which the imagesignal is read from the image sensor 11 shown in FIG. 6E.

Therefore, in order to output the partial image 7 of the specific areaPa with a delay, the timing adjustment unit 16 adjusts the timing ofreading the image signals from the image sensor 11 and the timing ofreading the partial image from the image storage portion 132.

FIGS. 7A to 7E show an example of timing adjustment when displaying apartial image in an enlarged scale in the PinP mode.

FIGS. 7A, 7D, and 7E show an example of the sensor output timing atwhich the image sensor 11 outputs image signals and the camera outputtiming at which the imaging device 1 outputs image signals to thedisplay device 2 similarly to FIGS. 6A, 6D, and 6E. However, the amountof timing adjustment when displaying the partial image in an enlargedscale in the PinP mode is not determined uniformly.

FIG. 7B shows an example (first case) in which the whole image 6 isdisplayed in a reduced scale in the specific area Pa, and the partialimage 7 is displayed on the full screen.

FIG. 7C shows an example (second case) in which the partial image 7 isdisplayed in a reduced scale in the specific area Pa, and the wholeimage 6 is displayed on the full screen.

In this case, it is difficult to absorb the difference in display timingjust through timing adjustment, the timing is adjusted in a manner asshown in FIGS. 8A to 8D or FIGS. 9A to 9D.

FIGS. 8A to 8D show an example (first case) of timing adjustment when apartial image is displayed in an enlarged scale in the PinP mode. Thedisplay control unit 17 displays a second image extracted as a specificarea on the full PinP screen and displays a first image in the childscreen.

FIG. 8A shows an example of the timing at which the image sensor 11outputs image signals.

As described above, the effective 1V period is calculated by thefollowing equation.

Tie−Ti1=Effective 1V Period

FIG. 8B shows an example of an area in which the partial image 7 is set.

As described above, the specific area Pa in which the partial image 7 isset is variable. Moreover, the variable effective period of the partialimage 7 is calculated by the following equation.

Tim−Tin=Variable Effective Period

FIG. 8C shows an example of a screen displayed in the PinP mode.

In the PinP presentation mode, the whole image 6 is displayed in areduced scale in the specific area Pa inside the partial image 7 whichis displayed on the full screen. The timing adjustment unit 16 adjuststhe image writing timing using the timing adjustment function, and thearea selection unit 19 expands and outputs images from the image storageportion 132 based on an instruction of the timing adjustment unit 16. Inthis case, when the display starting timing of the partial image 7displayed in an enlarged scale is To1, and the display ending timing isToe, the effective 1V period is calculated by the following equation.

Toe−To1=Effective 1V Period

FIG. 8D shows an example of displaying the whole image 6 in an enlargedscale.

Here, when the display starting timing of the whole image 6 is Tp1, andthe display ending timing is Tp, the effective 1V period is calculatedby the following equation. Here, the display starting timing Tp1 and thedisplay ending timing Tpe are variable depending on the arrangementposition of the PinP screen.

Tpe−Tp1=Effective 1V Period

FIGS. 9A to 9D show an example (second case) of timing adjustment whendisplaying a partial image in an enlarged scale in the PinP mode. Thedisplay control unit 17 displays a second image extracted as a specificarea on the child PinP screen.

FIGS. 9A and 9B show an example of the sensor output timing and thecamera output timing similarly to FIGS. 8A and 8B.

FIGS. 9C and 9D show an example of displaying the whole image 6 on thefull screen.

As shown in FIG. 9C, when the display starting timing of the whole image6 is To1, and the display ending timing is Toe, the effective 1V periodis calculated by the following equation.

Toe−To1=Effective 1V Period

As shown in FIG. 9D, the display starting timing Tp1 and the displayending timing Tpe of the partial image 7 displayed in the specific areaPa are variable depending on the arrangement position of the PinPscreen.

Tpe−Tp1=Effective 1V Period

FIGS. 10A and 10B show an example in which the left and right imagesincluded in the whole image 6 read from the image sensor 11 are enlargedin the vertical and horizontal directions.

FIG. 10A shows an example of an area 9L in which a partial image on theleft side of the whole image 6 is included and an area 9R in which apartial image on the right side is included. A left-directional imagesignal corresponding to an image obtained by imaging the left side ofone direction is read from the area 9L. On the other hand, aright-directional image signal corresponding to an image obtained byimaging the right side of one direction is read from the area 9R.

FIG. 10B shows an example in which the image included in the area 9L andthe image included in the area 9R are enlarged.

The image conversion processor 131 expands and enlarges theleft-directional image included in the area 9L and the right-directionalimage included in the area 9R among the images read from the imagesensor 11. Moreover, the image conversion processor 131 writes theleft-directional image signal and the right-directional image signal tothe first area 132α so that the image displayed on the display unit 21based on the image signals read from the first area 132α is displayedwithout overlap in an enlarged or reduced scales in the vertical andhorizontal directions of the screen of the display unit 21. In thepresent embodiment, the converted left-directional image signal iswritten to the first split area 132α1, and the convertedright-directional image signal is written to the second split area132α2, whereby left and right images 10L and 10R are generated.

FIG. 11 shows an example of an image conversion process on the left andright images by the image conversion processor 131.

In this example, the timing at which images are extracted from the imagesensor 11 to thereby generate enlarged left and right images.

Image signals of the periods of Tinll′ to Tinlr′ and Tinrl′ to Tinrr′ inthe horizontal direction of the area 9L are read from the image sensor11. The image conversion processor 131 performs an imageexpansion/reduction process of continuously outputting the read imagesignals during an effective 1H period. Moreover, the image conversionprocessor 131 performs the same timing expansion/reduction process onthe left-directional image signal and the right-directional image signalin the vertical direction to thereby generate an image of one frameperiod.

FIGS. 12A and 12B show an example of enlarging the central imageincluded in the whole image 6 read from the image sensor 11.

FIG. 12A shows an example of an area 8 in which a partial image at thecenter of the whole image 6 is included. A one-directional image signalcorresponding to an image obtained by imaging an area in one direction(in the present embodiment, the front direction) is read from the area8.

FIG. 12B shows an example of enlarging the image included in the area 8in the horizontal direction.

The image conversion processor 131 expands and enlarges the imageincluded in the area 8 among the images read from the image sensor 11 inthe horizontal direction. Moreover, the image conversion processor 131writes the converted one-directional image signal to the second area132β to thereby generate a central image so that the central image isdisplayed on the display unit 21 as PinP on the left and right imagesshown in FIG. 10B.

FIGS. 13A to 13D show an example of timing adjustment when a partialimage is displayed in an enlarged scale on the display unit 21 in thePinP mode.

FIG. 13A shows an example of the sensor output timing at which the imagesensor 11 outputs image signals similarly to FIG. 6A.

FIG. 13B shows an example in which areas 8, 9L, and 9R are set in thewhole image 6.

In the example shown in FIG. 13C, similarly to FIG. 12B, in the imagestorage portion 132, the left-directional image signal is written to thefirst split area 132α1, the right-directional image signal is written tothe second split area 132α2, and the one-directional image signal iswritten to the second area 132β.

FIG. 13D shows an example of an image based on the one-directional imagesignal written to the second area 132β.

As described above, the image conversion processor 131 arranges thesecond area 132β so as to comply with the upper, central, and lowerparts within the screen displayed on the display unit 21 and writes theone-directional image signal to the second area 132β. Moreover, theimage conversion processor 131 enlarges the second area 132β in thehorizontal direction on the upper or lower part within the screen andwrites the one-directional image signal to the second area 132β. Thus,it can be understood that by expanding the image based on theone-directional image signal written to the second area 132β, acompressed image which is extracted as the area 8 is obtained.

FIG. 14 shows an exemplary top view of an automobile in which theimaging device 1 of the present embodiment is installed.

The imaging device 1 is installed in a bonnet portion on the front sideof an automobile and a trunk portion on the rear side, and the displaydevice 2 is installed in the driver's seat inside the vehicle. Theimaging devices 1 installed on the front and rear sides have an imagingangle of about 180°. In the present embodiment, the “front direction” ofthe imaging device 1 provided on the front side is referred to as “onedirection”, and the “rear direction” of the imaging device 1 provided onthe rear side is referred to as “one direction”.

FIGS. 15A to 15C show an example in which an image obtained by imagingthe front side of a vehicle is displayed on the upper part of thedisplay unit 21 in the PinP mode.

FIG. 15A shows an example of the full-screen presentation when no imageconversion or the like is performed.

In this example, an image 30 captured by the imaging device 1 providedon the rear side of the automobile is displayed. Since the lens used inthe imaging device 1 has a large radius of curvature, image distortionat the periphery of the display unit 21 is particularly great.Therefore, the user cannot easily identify a vehicle that is to befocused on and may misunderstand the positional relation such that theapproaching vehicle appears to be distant from the driver's vehicle.

FIG. 15B shows an example of images obtained by imaging the front, left,and right sides, included in the image 30.

In this example, areas 31L and 31R located on the left and right sidesof the image 30 are displayed in an enlarged scale and are selected assplit main screens. Moreover, an area 32 located at the center of theimage 30 is selected as the child screen which is displayed as a PinPscreen in FIG. 15C. The selection is automatically performed based onthe setting value read from the setting value storage portion 15.

In this way, during a transition period in which partial imagesextracted from the whole image before conversion, normally seen by thedriver are displayed in an enlarged scale, the bold frames of areas(areas 31, 31L, and 31R) corresponding to the enlarged portions aretemporarily displayed. The period necessary for displaying the frames isabout 0.5 to 5 seconds, and the bold frames being displayed calls theattention of the user looking at the child screen displayed as the PinPscreen. The respective setting values are determined before the imageconversion processor 131 writes the one-directional image signal to thesecond area 132β and the left-directional image signal and theright-directional image signal to the first area 132α. The imageconversion processor 131 adds information representing a specific areaspecified by the setting values and writes image signals read from theimage sensor 11 to the image storage portion 132 as they are. In thisway, when the driver looks at the image displayed on the display unit 21to check other approaching vehicles or the like or the situation on theleft and right sides, it is possible to prevent a situation in which thedriver fails to recognize a pedestrian in front of the vehicle.

FIG. 15C shows an example in which a main screen and a child PinP screenare displayed together on the display unit 21.

Before displaying an image on the display unit 21, the image conversionprocessor 131 removes image distortion in the areas 31L, 31R, and 32.Moreover, the left and right images in the areas 31L and 31R shown inFIG. 15B are enlarged, and the left-directional and right-directionalimages are displayed on left and right screens 33L and 33R splitting themain screen. Moreover, the display control unit 17 which operates inaccordance with the operation of the user can display theone-directional image by changing the position and size of a childscreen 34 which is displayed as a PinP screen.

In this way, by displaying the images obtained by imaging the left andright sides of the vehicle in an enlarged scale, the user can recognizeother vehicles approaching the driver's vehicle. Moreover, by displayingthe child screen 34 in which the image included in the area 32 isconverted into a reduced scale on the upper part of the screen as a PinPscreen, the driver can easily understand the situation of thesurrounding of the driver's vehicle. Moreover, the driver can change thesize of the child screen 34 displayed as a PinP screen in an enlarged orreduced scale in the direction indicated by the broken arrows. By doingso, even when an automobile or the like included in the left or rightscreen 33L or 33R approaches the driver's vehicle and appears in anenlarged scale, the display of the child screen 34 is not impaired.

FIG. 16 shows an example in which an image obtained by imaging the frontside of a vehicle is displayed in a PinP mode on the lower part of thedisplay unit 21.

In this example, the child screen 34 in which a reduced image isdisplayed in a PinP mode is set in the central lower part of mainscreens 35L and 35R in which the left and right images are displayed.The size of the image displayed in the child screen 34 can be changed byenlarging or reducing the child screen 34 in the direction indicated bythe broken arrows similarly to the image shown in FIG. 15C. In this way,even when an automobile or the like included in the images displayed inthe main screens 35L and 35R approaches the driver's vehicle and appearin an enlarged scale, the display of the child screen 34 is notimpaired.

FIG. 17 shows an example in which an image obtained by imaging the frontside of a vehicle is expanded in the horizontal direction and displayedon the upper part of the display unit 21 in a PinP mode.

In this example, the image of an area 31 is displayed in a PinP mode ina child screen 37 which is arranged on the upper part of the screen ofthe display unit 21 and which is compressed in the vertical directionand expanded in the horizontal direction. Since the driver can easilyobserve the front side, the visibility is not impaired even when thechild screen 37 is compressed in the vertical direction. Moreover, mainscreens 36L and 36R displaying the split left and right main screens arearranged below the child screen 37, and the left-directional andright-directional images are displayed in the respective main screens inan enlarged scale. In this case, even when an automobile or the likeincluded in the images displayed in the main screens 36L and 36Rapproaches the driver's vehicle and appear in an enlarged scale, thedisplay of the child screen 37 is not impaired.

In this way, the image captured on the front or rear side of anautomobile is inserted into the child screen which is displayed as aPinP screen. In this case, the arrangement position of the child screencan be appropriately changed to the following position in relation tothe whole screen in accordance with the installed state of the imagingdevice 1, the screen size of the display unit 21, and the purpose ofimproving the visibility of the driver. That is, the child screen can beappropriately arranged at the following positions: a) the central upperpart of the screen; b) the central lower part of the screen; c) theupper horizontal part of the screen; and d) the lower horizontal part ofthe screen.

According to the imaging system 5 of the exemplary embodiment describedabove, it is possible to extract parts of an image read from the sameimage sensor 11 and display any one of a whole image or a partial imagein a child screen as a PinP screen. However, in order to display thewhole image and the partial image, it is necessary to adjust the timingbetween image conversion and displaying of images in two ways. Thus,when realizing the PinP presentation function as the child PinP screen,a memory area in which information of images displayed in the childscreen is stored is secured in the image storage portion 132 as thesecond area 132β, and images acquired from the full screen are insertedin advance into the second area 132β. In this way, it is not necessaryto adjust the timing during the process of outputting the full screenand the child screen in two ways. In this way, the imaging device 1disposes the images of the previous frame in a reduced scale in anaddress area corresponding to the PinP display position in thesequential readout address of “full image memory”, used when displayingthe full screen.

Moreover, the imaging device 1 has a function of arranging a childscreen within an image output by the imaging device 1 having the PinPfunction. Moreover, under the control of the imaging device 1, the imageobtained by imaging the left and right sides of a vehicle is displayedenlarged on the full screen, and the image of the front side of thevehicle is displayed in the child screen. In this way, since it ispossible to identify the moving direction of the vehicle and an objectapproaching the driver's vehicle, the driver can determine whetherforward or backward movement of the driver's vehicle will obstruct thepath of other vehicles or the like in advance of moving the vehicle. Inthe related art, when driving a vehicle across a road at a right angleor pulling the vehicle out from a parking garage, it was difficult forthe driver to observe other vehicles or pedestrians approaching from theleft and right sides from the driver's seat. However, it is possible toobserve the left and right-side images and the front and rear-sideimages of the vehicle in an enlarged scale from a wide-angle image onone screen displayed on the display unit 21 at a low cost. Moreover, itis possible to provide the imaging system 5 at a low cost which enablesthe driver to safely drive a vehicle from intersections or a parkinggarage with poor visibility.

Moreover, the output of the respective conversion images are updatedevery two frames. Thus, images obtained at different times are combinedand output to the display unit 21. Therefore, it is possible to outputimages of every frame to the display unit 21 without decreasing theframe rate of images output by the image sensor 11. Moreover, since theimaging device 1 outputs images to the display device 2, it is possibleto construct the imaging system 5 so as to output images continuously insynchronization with a reference vertical synchronization timing signalsuch as the NTSC analog output.

As described above, the imaging device 1 can capture a wide-angle imageas required in an in-vehicle surrounding monitoring camera having thePinP function. From a wide-angle image, it is difficult to understandthe details of a local image. In particular, even when a specific areathat the user wants to focus on is present in the wide-angle image, ittakes a lot of time for the user to realize the situation or the usercannot understand the details from the image as it was. In addition, itis important for the driver to detect a danger in an early stage whiledriving. In view of such a demand, by providing a function in which theimaging device 1 sets a corresponding partial image in the specific areaPa from the wide-angle image captured by the imaging device 1 in advanceand displaying the corresponding image in the child screen or the likeas the PinP screen, the driver can easily understand the situation. Bydoing so, since the driver can observe a specific focusing area in moredetail together with the whole image, the driver can easily and quicklyunderstand the surrounding situation.

Moreover, the imaging device 1 has the PinP function of arranging achild screen within the same screen output by the imaging device 1.Thus, the driver or the like can designate a partial image of an areathat is difficult for the driver or the like to observe from the wholeimage with naked eyes in the child screen. Alternatively, it is alsopossible to display the whole image in the child screen and display adetailed enlarged image of a specific area on the full screen. Moreover,by transmitting the whole image and the partial image to the displaydevice 2 so as to be inserted into the same screen as the PinP screensubstantially at the same time, the driver can understand the whole areaof the surroundings and the details of the specific area at the sametime.

Moreover, since the images are updated every two frames, the imagequality adjustment unit 12 adjusts the image qualities of areas whichare read at different times and are associated with different imagingareas to the optimum quality, whereby the imaging device 1 can outputthe respective conversion images. Moreover, since the PinP presentationmethod is realized by alternately writing frames to the image storageportion 132, it is possible to obtain an additional advantage that it ispossible to optimize the exposure control for each area. In the relatedart, when areas of which the brightness changes greatly are displayed inthe PinP mode, some images may appear too dark or too bright. However,by adjusting the image quality of images displayed in each area to theoptimum quality, it is possible to adjust the brightness of images basedon image signals written to the respective areas in units of frames.Therefore, the imaging device 1 can output images which are generallyeasily understood by the driver.

In the existing in-vehicle imaging devices for monitoring thesurroundings, monitoring areas are set in advance for the purpose ofobserving specific areas that are set in the course of manufacturing.Although there are products in which visual management areas can bedesignated before shipment from factories, there is no product in whichmonitoring areas can be designated taking low-visibility situationassociated with the respective users' driving environment intoconsideration. On the other hand, according to the imaging system 5,since the driver can designate low-visibility areas under the driver'sdriving environment and maintain the designated areas, it is possible tomonitor the surroundings effectively, for example, when parking in anarrow space at the driver's home.

<2. Modified Example>

As shown in FIGS. 4A to 4C, the present disclosure can be applied to notonly a configuration in which the display of the PinP screen iscontrolled using one image storage portion 132 but also a configurationin which a plurality of image storage portions 132 are provided so as tocorrespond to the number of areas.

The present disclosure is not limited to the above-described embodiment,but various applications and modifications can be made without departingfrom the spirit of the present disclosure described in the claims.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2010-191376 filed in theJapan Patent Office on Aug. 27, 2010, the entire content of which ishereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1-10. (canceled)
 11. A display control apparatus for a vehiclecomprising: a control unit configured to control a display in a panoramaview mode for displaying an entire image and in a multi-image view modefor displaying a left-directional image, a right-directional image and aone-directional image; and an output unit configured to output imagedata in the panorama view mode or the multi-image view mode.
 12. Thedisplay control apparatus for a vehicle according to claim 11, whereinthe entire image is captured by a single camera which includes awide-angle lens.
 13. The display control apparatus for a vehicleaccording to claim 12, further comprising an image conversion unitconfigured to remove image distortion in the left-directional image andthe right-directional image and the one-directional image.
 14. Thedisplay control apparatus for a vehicle according to claim 11, whereinthe control unit is configured to control the display to generate arectangular image as the one-directional image.
 15. The display controlapparatus for a vehicle according to claim 14, wherein the rectangularimage exhibits a size that is changeable by a user.
 16. The displaycontrol apparatus for a vehicle according to claim 11, wherein theone-directional image is displayed on an upper portion of the display.17. The display control apparatus for a vehicle according to claim 16,wherein the one-directional image is displayed as a horizontallyexpanded image.
 18. The display control apparatus for a vehicleaccording to claim 17, wherein the left-directional image and theright-directional image are displayed in display areas below a displayarea of the one-directional image and in a scale relatively enlargedwith respect to the scale of the display of the one-directional image.19. The display control apparatus for a vehicle according to claim 11,wherein the display of the panorama view mode and display of themulti-image view mode are switchable by a user.
 20. A method forcontrolling a display for a vehicle, comprising: controlling the displayin a panorama view mode to display an entire image and in a multi-imageview mode to display a left-directional image, a right-directional imageand a one-directional image; and outputting image data to the displayfor the panorama view mode or the multi-image view mode.
 21. The methodof claim 20, comprising capturing the entire image by a single camerawhich includes a wide-angle lens.
 22. The method of claim 21, furthercomprising removing image distortion in the left-directional image andthe right-directional image and the one-directional image.
 23. Themethod of claim 20, wherein the one-directional image is displayed as arectangular image.
 24. The method of claim 23, wherein the rectangularimage exhibits a size that is changeable by a user.
 25. The method ofclaim 20, further comprising displaying the one-directional image on anupper portion of the display.
 26. The method of claim 25, furthercomprising displaying the one-directional image as a horizontallyexpanded image.
 27. The method of claim 26, further comprisingdisplaying the left-directional image and the right-directional image indisplay areas below a display area of the one-directional image and in ascale relatively enlarged with respect to the scale of the display ofthe one-directional image.
 28. The method of claim 20, furthercomprising switching, by a user, between the display of the panoramaview mode and display of the multi-image view mode.
 29. A displaycontrol system for a vehicle comprising: an image pickup device; andcontrol apparatus, including a control unit configured to control adisplay in a panorama view mode for displaying an entire image and in amulti-image view mode for displaying a left-directional image, aright-directional image and a one-directional image; and an output unitconfigured to output image data in the panorama view mode or themulti-image view mode.
 30. The system of claim 29, wherein the imagepickup device comprises a single camera which includes a wide-angle lensto capture the entire image.
 31. The system of claim 30, furthercomprising an image conversion unit configured to remove imagedistortion in the left-directional image and the right-directional imageand the one-directional image.
 32. The system of claim 29, wherein thecontrol unit is configured to control the display to generate arectangular image as the one-directional image.
 33. The system of claim32, wherein the rectangular image exhibits a size that is changeable bya user.
 34. The system of claim 29, wherein the one-directional image isdisplayed on an upper portion of the display.
 35. The system of claim34, wherein the one-directional image is displayed as a horizontallyexpanded image.
 36. The system of claim 35, wherein the left-directionalimage and the right-directional image are displayed in display areasbelow a display area of the one-directional image and in a scalerelatively enlarged with respect to the scale of the display of theone-directional image.
 37. The system of claim 29, wherein the displayof the panorama view mode and display of the multi-image view mode areswitchable by a user.